The // SAFETY assertion is not just a regular comment that explains what happens, but it needs to reason why and how the safety assumptions (specified in the function docs) are upheld.

Would also be good to have an empty line before it.

This is unnecessarily hard to understand, could you just write a

if vec.is_empty() {
    return Self::new();
}

instead?

So something like this?

if vec.is_empty() {
    return Self::new();
}
let len = vec.len();
let mut ret = Self::default_with_size(len);

I don't know, but it doesn't save enough space to justify default_with_size and stand out with above Froms.

Yeah, maybe I was too optimistic with the code reuse here... Otherwise it's OK to not have default_with_size. More important than a bit of duplication or fewer lines of code is readability (which is problematic with the match version).

Comments should express more than the function name, or be omitted.

Here, you could mention something like